Apparatus and method for generating interpolated image data

ABSTRACT

Blur-free, high-quality frame image data are generated from field image data. Image data of odd-numbered field images and image data of even-numbered field images are alternately applied every 1/60 of a second. The even field image data are applied to a first memory, and data of odd field images located immediately before and immediately after the even field image are applied to respective ones of a second memory and a third memory. Windows are set on the respective images and whether or not motion has occurred between the windows of the two odd field images is detected. If motion has not occurred, data of a pixel missing in the even field is generated using the pixel, of an odd field, at a position corresponding to this pixel missing in the even field and for which data are to be generated. If motion has occurred, the data of a pixel missing in the even field image is generated using pixels above and below the missing pixel in the even field and for which data are to be generated.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to an apparatus and method for generatinginterpolated image data, particularly an apparatus and method forconverting a field image, which comprises pixels of odd or even lines,to a frame image.

2. Description of the Related Art

A digital video tape recorder (DVTR) senses the image of a subject usinga solid-state electronic image sensing device such as a CCD, converts avideo signal, which represents the image of the subject obtained byimage sensing, to digital image data and records the digital image dataon magnetic tape. In a DVTR, the general practice when recording amoving picture is to perform interlaced readout of the CCD at a fixedperiod and alternately obtain a field image comprising pixels of oddlines and a field image comprising pixels of even lines. This makes itpossible to shorten the image sensing period so that a smoothly movingimage can be recorded even if the subject is moving rapidly.

In a case where a moving picture is reproduced, a field image comprisingpixels of odd lines and a field image comprising pixels of even linesare alternately applied, whereby a frame image is constructed anddisplayed on a television. One frame image is constructed from twoconsecutive field images. For example, a first frame image isconstructed from a first field image and a second field image, a secondframe image is constructed from a third field image and a fourth fieldimage, a third frame image is constructed from a fifth field image and asixth field image, and a fourth frame image is constructed from aseventh field image and an eighth field image.

In order to meet the demand for printout of a desired scene, a DVTRhaving a still-picture reproduction function in addition to amoving-picture reproduction function has been developed. However, inDVTRs in which a field image obtained in moving-picture recording isprinted out as is, vertical resolution is poor and a print having a highpicture quality is not obtained because the field image is composed ofpixels of odd lines or even lines only.

When a frame image is constructed from two consecutive field images inorder to obtain a print having a high picture quality, a temporal offsetdevelops between the two field images when the images are sensed.Consequently, if the subject is moving, the print obtained will not beclear because of blurring of the image. In order to solve this problem,a method is available through which pixel data are interpolated usingimage data of pixels immediately above and below a pixel for which imagedata are to be generated in the field image For example, see thespecification of Japanese Patent Application Laid-Open (KOKAI) No.63-187785).!. However, a disadvantage with this method is that diagonallines develop jaggies.

In order to solve this problem, directions having strong correlation aredetected in the field image about the pixel for which image data is tobe generated, interpolation is performed using the image data of pixelspresent along the detected directions, and a frame image is generatedfrom the field image. For example, see the specification of JapanesePatent Application Laid-Open (KOKAI) No. 4-366894).!

With this method, however, the improvement in vertical resolution andthe elimination of jaggies in the diagonal direction are not necessarilysatisfactory.

Furthermore, a technique is available in which a frame image is obtainedby changing the interpolation method in dependence upon motion of thesubject utilizing the fact that, in the recording of an image, a methodof coding performed when the subject is at rest differs from thatperformed when the subject is moving. For example, see the specificationof Japanese Patent Application Laid-Open (KOKAI) No. 4-86185).! Thisrequires judging whether the subject is moving or is at rest. In orderto make this judgment, the general practice is to calculate a motionvector (using a motion vector method). However, calculating motionvectors involves a great deal of computation. In addition, using themotion vector method to obtain a high-quality still picture isimpractical.

SUMMARY OF THE INVENTION

An object of the present invention is to make it possible to generate aframe image for a high-quality still picture, which has little imageoffset, from a field image that is for producing a moving picture.

According to a first aspect of the present invention, the foregoingobject is attained by providing an apparatus in which, when data ofconsecutive first, second and third field images are given among aseries of image data obtained by interlaced scanning wherein odd fieldimages comprising pixels on odd lines and even field images comprisingpixels on even lines repeat in alternating fashion, interpolated imagedata are generated representing pixels on even lines in a case where thesecond field is an odd field and pixels on odd lines in a case where thesecond field is an even field in order to convert the second fieldimage, which is located between the first field image and the thirdfield image, to a frame image. A motion detector detects whether motionhas occurred between images in windows set at corresponding positions onrespective ones of the first field image and third field image. A firstinterpolated image data generator which, when motion has been detectedby the motion detecting means, generates image data of a pixel at aposition corresponding to pixels in the windows based upon image data ofpixels, of the second field, neighboring this pixel from above andbelow. A second interpolated image data generator which, when no motionhas been detected by the motion detecting means, generates image data ofa pixel at a position corresponding to pixels in the windows based uponimage data of a corresponding pixel in at least one of the first fieldimage and third field image.

The present invention according to the first aspect thereof alsoprovides a method of generating interpolated image data. Specifically,the invention provides a method in which, when data of consecutivefirst, second and third field images are given among a series of imagedata obtained by interlaced scanning wherein odd field images comprisingpixels on odd lines and even field images comprising pixels on evenlines repeat in alternating fashion, interpolated image data aregenerated representing pixels on even lines in a case where the secondfield is an odd field and pixels of odd lines in a case where the secondfield is an even field in order to convert the second field image, whichis located between the first field image and the third field image, to aframe image. The method includes detecting whether there is motionbetween images in windows set at corresponding positions on respectiveones of the first field image and the third field image, performingfirst interpolated image data generating processing, when motion hasbeen detected, for generating image data of a pixel at a positioncorresponding to pixels in the windows based upon image data of pixels,of the second field, neighboring this pixel from above and below, andperforming second interpolated image data generating processing, when nomotion has been detected, for generating image data of a pixel at aposition corresponding to pixels in the windows based upon image data ofa corresponding pixel in at least one of the first field image and thirdfield image.

The motion detection processing and the first interpolated image datagenerating processing or second interpolated image data generatingprocessing are repeated at each position of the windows while thepositions of the windows are successively shifted on the first fieldimage and third field image horizontally and vertically in increments ofwindow size, and results of processing are combined with the secondfield image to obtain a frame image.

Motion can be detected to have occurred when the value of a sum isgreater than a predetermined threshold value, wherein the sum isobtained by summing, within the windows, the differences betweenabsolute values of the levels of corresponding pixels in the window ofthe first field image and the window of the third field image. It hasbeen ascertained from the results of tests that the threshold value inthis case desirably is a value obtained by multiplying the number ofpixels constituting the window by a numerical value of 1˜3 when theabove-mentioned pixel level is represented by eight bits. It isespecially preferred that the number of pixels constituting the windowby multiplied by a numerical value of 1.5˜2.5 when the pixel level ispresented by eight bits.

It is preferred that the above-mentioned window be longer in thevertical direction than in the horizontal direction. For example, thenumber of pixels in the vertical direction desirably is twice the numberof pixels in the horizontal direction. The reason for this is asfollows: A field image consists solely of pixels on even lines or oddlines. This means that if the window is set to have the same lengths inthe horizontal and vertical directions, the number of pixels in thevertical direction will be less than the number of pixels in thehorizontal direction and it will be difficult to detect motion if thesubject moves in the vertical direction.

In accordance with the first aspect of the present invention, motionbetween the images in windows set at corresponding positions on firstand third field images is detected. If motion has been detected, thepixels in the window of the first field image are considered to have agreater correlation with the pixels in the window of the second fieldimage than with the pixels in the window of the third field image.Interpolated image data are generated using the image data of pixels inthe second field image, these pixels residing immediately above andbelow in the region corresponding to the window. The interpolated imagedata are generated using pixels having a strong correlation betweenthem. If there is no motion, the correlation between a pixel in thewindow of the first field image and a pixel in the window of the thirdfield image is construed to be strong. Accordingly, interpolated imagedata of the second field are generated using image data of a pixel ofthe first field and image data a pixel of the second field, these pixelsbeing located at corresponding positions in the frames.

In accordance with the first aspect of the invention, a pair of pixelshaving strong correlation between them is found from a pair of pixels,one of which is contained in the image of the first field and the otherof which is contained in the image of the third field, or a pair ofpixels both of which are contained in image of the second field.Interpolated image data are generated from these pixels having strongcorrelation. Since a pair of pixels having strong correlation representsthe same pixel, a frame image represented by the interpolated image datagenerated and the image data of the second field will have smooth edges.The second field image comprises odd lines or even lines and has acomparatively low resolution. However, the frame image obtained byperforming pixel interpolation exhibits a high vertical resolution. Astill picture having a high picture quality is obtained as a result.

Further, since motion vectors are not used, fewer calculations arenecessary in comparison with a case where a frame image is obtainedutilizing the motion vector method.

The processing for generating the first interpolated image data can beimplemented by extracting image data representing pixels above and belowa pixel missing in the second field image, as well as image datarepresenting pixels above and below pixels which should be present atpositions neighboring the missing pixel in the horizontal direction,obtaining values of sums of the image data representing these pixels,determining whether values of the sums obtained are increasing ordecreasing with regard to pixels in the horizontal direction, generatingdata representing the missing pixel using the data representing thepixels above and below the missing pixel when the values of the sums areincreasing or decreasing, calculating correlation values between pixelsin three directions, namely of pixels above and below and diagonallyabove and below the missing pixel when the values of the sums are notincreasing or decreasing, and generating data representing the missingpixel using data representing pixels which give the highest correlationvalue.

When the above-mentioned values of the sums are increasing or decreasingin the horizontal direction, the image of this portion is a step-shapedportion. In such case, there are instances where the step-shaped portionis emphasized when the data representing the missing pixel aregenerated. This can cause a disturbance in the image.

An interpolated image data generating apparatus that is availableextracts image data representing two pixels in each of three directions,namely two pixels above and below a missing pixel, pixels at the upperright and lower left of the missing pixel and pixels at the upper leftand lower right of the missing pixel, calculates the correlation valuesbetween these pixels and generates data representing the missing pixelfrom two pixels having the highest correlation. In this case, even whenpixels having slightly different levels reside above and below themissing pixel and the levels of the pixels at the upper right and lowerleft of the missing pixel or at the upper left and lower right of themissing pixel are zero, the correlation between the two pixels at theupper right and lower left of the missing pixel and or between the twopixels at the upper left and lower right of the missing pixel is high.As a result, the missing pixel is generated using these two pixels.Consequently, an unnatural image different from the actual image isobtained.

In the description given above, interpolated image data of the missingpixel is generated using two pixels that give the highest correlationwhen the above-mentioned mentioned values of the sums are increasing ordecreasing with regard to the horizontal direction, and interpolatedimage data of the missing pixel is generated using pixels above andbelow the missing pixel when the above-mentioned values of the sums arenot increasing or decreasing with regard to the horizontal direction. Asa result, any step-portion that might exist is not emphasized and animage close to the actual image is obtained.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the electrical construction of anapparatus for generating frame image data according to an embodiment ofthe present invention;

FIG. 2 illustrates the manner in which odd-numbered field images andeven-number field images appear in alternating fashion;

FIG. 3 illustrates, in the form of a plane, the correspondingrelationship between pixels for generating frame image data;

FIGS. 4a and 4b illustrate, in the form of a time series, thecorresponding relationship between pixels for generating frame imagedata;

FIG. 5 is a block diagram showing the electrical construction of acircuit for calculating interpolated pixel values;

FIG. 6 illustrates a pixel block for generating the data of a missingpixel;

FIG. 7 is a flowchart illustrating a processing procedure for generatingthe data of a missing pixel; and

FIG. 8 is a block diagram showing the electrical construction of adigital video tape recorder.

DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1 is a block diagram showing the electrical construction of anapparatus for generating frame image data from field image dataaccording to an embodiment of the present invention. In a case where thedata of at least three field images, namely first, second and thirdfield images, are contiguous in the order mentioned and odd- andeven-numbered field images appear alternately, the apparatus generatesthe frame image data from the second field image data.

As shown in FIG. 2, the given image data are such that odd field images(first, third, fifth and seventh field images), each of which comprisespixels on odd-numbered lines, and even field images (second, fourth,sixth and eighth field images), each of which comprises pixels oneven-numbered lines, appear alternately. A case will be described inwhich a frame image is generated by interpolating image data of thesecond field when the first, second and third field images in FIG. 2 aregiven consecutively in the order mentioned.

The apparatus for generating the frame image data includes three fieldmemories 11, 12 and 13. Changeover of a changeover switch 10 iscontrolled in such a manner that field image data that generates frameimage data by being interpolated is stored in a field memory 12 amongthe field memories 11, 12 and 13, and field image data before and afterthis field image data are stored in the field memory 11 or field memory13, respectively. In this example, it is assumed that the image data ofthe first field is stored in the field memory 11 and that the image dataof the third field is stored in the field memory 13. The image data ofthe second field is stored in the field memory 12.

Though the changeover switch 10 is illustrated as being a contactswitch, in actuality the switch 10 is implemented in the form of asemiconductor contactless switch (a gate circuit, etc.) as a matter ofcourse.

FIG. 3 illustrates pixel groups of eight pixels in the row (vertical)direction and four pixels in the column (horizontal) direction extractedfrom the first, second and third field image data through windows W1, W2and W3. Since the first field image data and the third field image dataare image data of odd-numbered fields, image data representing thepixels on odd-numbered rows are present. Since the second field imagedata are image data of an even-numbered field, image data representingthe pixels on even-numbered rows are present. Pixels at which image dataexist are indicated by hatching in FIG. 3. Further, FIG. 3 is drawn inthe form of a plane in order to facilitate an understanding of the pixelarrays.

In the apparatus of FIG. 1 for generating frame image data, it isdetermined whether motion has occurred between the image in the windowW1 of the first field and the image in the window W3 of the third field.If there is no motion, this means that the image in window W3 isunchanged from the image in window W1. It is construed, therefore, thatthe image in window W2 of the second field image has a strongcorrelation to the image in window W1 of the first field and the imagein window W3 of the third field image, and the pixels of the missingodd-numbered rows in the second field image are generated using theimage data of the corresponding pixels of the first and third fields. Ifmotion does occur, it is considered that there has been a change ofscene or the like. Accordingly, it is construed that the image in windowW2 of the second field image does not have a strong correlation to theimage in window W1 of the first field image or to the image in window W3of the third field image, and the pixels of the missing odd-numberedrows in the second field image are generated using the image data of thepixels that are present in the window W2 of the second field image. Thedetermination as to whether motion has occurred between the window W1 ofthe first field image and the window W3 of the third field image can beimplemented by calculating the difference between the absolute values ofthe image data of corresponding pixels in the windows W1 and W3 withregard to all pixels present in the windows W1 and W3, and judgingwhether the sum obtained by summing all of these differences is greaterthan a predetermined threshold value. The threshold value preferably isone obtained by multiplying the number of pixels constructing a windowby a numerical value of 1˜3, especially a numerical value of 1.5˜2.5, ina case where the level of the image data representing a pixel isrepresented by eight bits.

As one example, a case will be described in which data representing apixel (2,10) of the second field is generated by interpolationprocessing. FIGS. 4a and 4billustrate the pixels of the second column inwhich the pixel (2,10) in window W2 of the second field image exists, aswell as the pixels in window W1 of the first field image and in windowW3 of the third field image corresponding to the pixels of the secondcolumn of window W2. Pixels at which data exist are indicated by thecircle marks in FIGS. 4a and 4b, and pixels at which there are no dataare indicated by the "x" marks. As will be understood from FIG. 3 andFIGS. 4a, 4b, pixels exist in the odd rows of the first and third fieldimages and in the even rows of the second field image.

It is determined whether motion has occurred between the image in windowW1 of the first field image and the image in window W3 of the thirdfield image. If motion has not occurred, the mean of the image data ofpixel (1,10) of the first field image and the image data of pixel (3,10)of the third field image (the pixels (1,10) and (3,10) are at thepositions corresponding to the pixel (2,10) of the second field image)is calculated and the mean image data are adopted as the image data ofthe pixel (2,10) of the second field image, as shown in FIG. 4a. Ifmotion has occurred, on the other hand, the mean of the image data ofpixels (2,6) and (2,14) above and below the pixel (2,10) of the image ofthe second field is calculated and these mean image data are adopted asthe image data of the pixel (2.10) of the second field image, asillustrated in FIG. 4b.

In the case where there is no motion, either the image data of pixel(1,10) of the first field image or the image data of pixel (3,10) of thethird field image (the pixels (1,10) and (3,10) are at the positionscorresponding to the pixel (2,10) of the second field image, asmentioned above) may be adopted as the image data of the pixel (2,10) ofthe second field image. In such case, it is preferred that use be madeof the image data of the pixel of the image constituting the same frame.For example, with regard to the image of the second field, the imageconstituting the same frame is the image of the first field; hence, useis made of the image data of pixel (1,10) of the image of the firstfield.

All of the image data of pixels of the missing even rows in the windowW2 of the second field are generated in the same manner as the imagedata of pixel (2,10). The processing for generating the image data ofmissing pixels is carried out with regard to the entirety of the secondfield image, whereby the image data of the second field image becomesthe frame image data.

The window is not limited to a rectangular window the length of whichextends in the row direction. It is permissible to use a square window.Further, the window is not limited to one having eight pixels in the rowdirection and four pixels in the column direction. The window can be setat will to pixel arrays of 4×4 pixels, 8×8 pixels, 16×16 pixels, 16pixels in the row direction by eight pixels in the column direction,four pixels in the row direction by two pixels in the column direction,etc.

With reference again to FIG. 1, window circuits 14, 15 and 16 set theaforementioned windows W1, W2 and W3 on the field image data that havebeen stored in the field memories 11, 12 and 13. From among the windowsW1, W2 and W3 that have been set by the window circuits 14, 15 and 16,the image data of pixels contained in the window W1 of the first fieldimage and the image data of pixels contained in the window W3 of thethird field image are applied to a motion detecting circuit 17, wherebyit is determined whether motion has occurred between the image in windowW1 and the image in window W3. A signal representing the result ofdetection performed by the motion detecting circuit 17 is applied to afirst interpolated pixel value computation circuit 20 and to a secondinterpolated pixel value computation circuit 21.

The windows W1, W2 and W3 set by the window circuits 14, 15, and 16 arescanned horizontally and vertically so as not to overlap one another onthe same field image. Interpolated image data are calculated at eachscanning position. Accordingly, interpolated image data are generatedwith regard to the pixels on all odd rows of the second field image.

The window circuits 14, 15 and 16 can each be constructed from aplurality (e.g., four) of cascade-connected line memories. The motiondetecting circuit 17 can be constructed from a subtractor circuit, asquaring circuit, an adder circuit and a level discriminator circuit. Ina case where the above is implemented by software, either the setting ofthe windows or the detection of motion or both of these operations wouldbe implemented by a computer so programmed.

The image data of the pixels contained in the windows W1 and W3 set bythe window circuits 14 and 16 are also applied to the secondinterpolated pixel value computation circuit 2 1. The image data of thepixels contained in the window W2 set by the window circuit 15 areapplied to the first interpolated pixel value computation circuit 20.

The first interpolated pixel value computation circuit 20 is providedwith image data representing the pixels above and below a missing pixel.This is data of the same field image as that of the missing pixel, asshown in FIG. 4b. The second interpolated pixel value computationcircuit 21 is provided with image data representing the pixels at thepositions corresponding to that of the missing pixel of the second fieldimage, this being the pixel for which interpolated image data is to begenerated. The provided image data are data of the first field image anddata of the third field image that come before and after the secondfield image having the missing pixel for which the interpolated imagedata in to be generated, as shown in FIG. 4a. The first interpolatedpixel value computation circuit 20 operates and the second interpolatedpixel value computation circuit 21 ceases operating when the motiondetection signal provided by the motion detecting circuit 17 isindicative of motion. The second interpolated pixel value commutationcircuit 21 operates and the first interpolated pixel value computationcircuit 20 ceases operating when the motion detection signal provided bythe motion detecting circuit 17 indicates no motion. Accordingly,results of interpolation are output from either circuit 20 or 21,depending upon whether there is motion between windows W1 and W3.

When there is no motion, the interpolated pixel value computationcircuit 21 may be provided with the data of the pixel of the first fieldimage or data of the pixel of the third field image, which pixel is atthe position corresponding to the pixel of the second field image to begenerated. In such case the interpolated pixel value computation circuit21 would not calculate a mean and would merely allow the data to passthrough.

The image data output by the interpolated pixel value computationcircuit 20 or 21 is applied to a frame memory 22. The image data of thesecond field image that has been stored in the field memory 12 also isapplied to the frame memory 22. By generating data representing allpixels of missing even-numbered rows of the second field image in theinterpolated pixel value computation circuit 20 or 21 and applying thesedata to the frame memory 22, these data are combined with the data ofthe second field image already stored in the frame memory 12, therebyforming a frame image.

In the description given above, the interpolated pixel value computationcircuits 20 and 21 are averaging circuits for calculating the mean valueof input image data representing two pixels. However, an arrangement maybe adopted in which the first interpolated pixel value computationcircuit 20 is made to generate the interpolated image data in a mannerdescribed below. Further, an arrangement may be adopted in which theoutputs of the window circuits 14, 15 and 16 are applied to a singleinterpolated pixel value computation circuit which then proceeds tocalculate the interpolated pixel value.

FIG. 5 is a block diagram showing the electrical construction of theinterpolated pixel value computation circuit. FIG. 6 illustrates a pixelblock composed of 3×3 pixels. Pixels for which image data exist areindicated by hatching in FIG. 6 in the same manner as in FIG. 3. In FIG.6, image data exist for pixels A, B, C, F, G and H but there is no imagedata for pixels D, X and E.

In a case where a pixel for which data are to be generated isrepresented by X, as shown in FIG. 6, the interpolated pixel valuecomputation circuit shown in FIG. 5 considers a pixel block of 3×3pixels centered on the pixel X and generates suitable image data of thepixel X by referring to the image data representing the pixels containedin this pixel block.

If, when pixels (A and F, B and G, C and H) above and below pixels forwhich there are no image data are added, the value of the sum increasesor decreases in the horizontal direction, the image of this portion is astep-shaped portion. If the arithmetic mean of the image data of thepixels above and below a pixel for which there are no image data areadopted as the image data of the missing pixel, the step-shaped portionwill be emphasized. On the other hand, if the correlation values of twopixels bracketing the pixel X are calculated and the arithmetic mean ofthe two pixels giving the highest correlation is adopted as the imagedata of the missing pixel, there are instances where the result is anunnatural image, as when the levels of the image data of the pixels A,F, C and H in the columns adjoining the column of the mixing pixel X areextremely low and the correlation high (e.g., when all the levels arezero) or when the levels of the image data of pixels B and G above andbelow the pixel X are high and the correlation low (e.g., when thelevels are the maximum level and near the maximum level).

The interpolated pixel value computation circuit shown in FIG. 5 is soadapted that even if the portion of the image for which data are to begenerated is a step-shaped portion, interpolated image data of a missingpixel are generated without emphasizing the step-shaped portion and insuch a manner that a natural image is obtained. The interpolated pixelvalue computation circuit shown in FIG. 5 finds two pixels deemed to bethe most favorable for generating the image data of a missing pixel andadopts the mean value of the image data of these two pixels as the imagedata representing the missing pixel.

FIG. 7 illustrates a procedure for generating the image data of amissing pixel. The level of image data representing a pixel is indicatedby the same character as that used to identify the pixel.

First, the sum Da of the image data of pixels A and F, the sum Xa of theimage data of pixels B and G and the sum Ea of the image data of pixelsC and H are calculated (step 60). The sums Da, Xa and Ea obtained arecompared and it is determined whether they are increasing (Da<Xa<Ea) ordecreasing (Da>Xa>Ea) in the horizontal direction (step 61). If the sumsare not increasing or decreasing ("NO" at step 61), then it is construedthat the peripheral portion of pixel X for which data are to begenerated is not a step-shaped portion and the mean value of the imagedata of pixels B and G above and below the pixel X is adopted as theimage data of the pixel X (step 67). If the sums are increasing ordecreasing ("YES" at step 61), then it is construed that the peripheralportion of pixel X for which data are to be generated is a step-shapedportion. If the mean value of the image data of pixels B and G above andbelow the pixel X were to be adopted as the image data of the pixel X,the step-shaped portion would be emphasized. Accordingly, the mean valueof the image data of the pair of pixels having the highest correlationof the two pairs of pixels lying diagonal to the pixel X is adopted asthe image data of pixel X.

The difference S1 (=|A-H|) between the absolute values of the image dataof pixel A to the upper left of pixel X and of the image data of pixel Hto the lower right of pixel X, the difference S2 (=|B-G|) between theabsolute values of the image data of pixel B above pixel X and of theimage data of pixel G below pixel X, and difference S3 (=|C-F|) betweenthe absolute values of the image data of pixel C to the upper right ofpixel X and of the image data of pixel F to the lower left of pixel Xare calculated (step 62). The combination of pixels having the highestcorrelation is judged based upon the results of calculating theseabsolute-value differences (steps 63 and 64). The mean of the image dataof the pixels giving the highest correlation is adopted as the imagedata of pixel X (step 65, 66 or 67).

With reference to the upper part of FIG. 5, one frame of digital imagedata representing luminance is stored in a field memory 31. Image datarepresenting the pixels (A, B, C, F, G and H) at the upper left, above,upper right, lower left, below and lower right of the pixel X for whichdata are generated are read out of the field memory 31 and the data areapplied to an interpolation-direction discriminator circuit 33 and amean-value computing circuit 34. The interpolation-directiondiscriminator circuit 33 executes the processing of steps 60˜64 shown inFIG. 7. When the best two pixels for generating the image data of pixelX are found in the interpolation-direction discriminator circuit 33, thesignals representing these two pixels are applied to the mean-valuecomputing circuit 34. The latter executes the processing (steps 65˜67 ofFIG. 6) for calculating the mean value of the image data of the twooptimum pixels. The data representing the result of calculation in themean-value computing circuit 34 are applied to a field memory 32, wherethe data are stored. By repeating the interpolation-directiondiscrimination processing in the interpolation-direction discriminatorcircuit 33 and the mean-value computation processing in the mean-valuecomputing circuit 34, all of the data representing the pixels of missinglines in one frame of field image data are generated and stored in thefield memory 32.

The image data that have been stored in the field memory 31 and theimage data that have been stored in the field memory 32 are image dataof mutually different lines in one frame of an image. For example, theimage data that have been stored in the field memory 31 are image dataof odd-numbered lines and the image data that have been stored in thefield memory 32 are image data of even-numbered lines. By combiningthese image data, therefore, frame image data of one complete frame areformed. The image data that have been stored in the field memories 31and 32 are applied to an adder circuit 35. As a result, the addercircuit 35 outputs one frame of frame image data.

With regard to R-Y and B-Y color-difference data, the data representingthe pixel of a missing row makes use of the image data representing thepixel lying above the missing pixel.

With reference to the center and lower parts of FIG. 5, one frame ofcolor-difference data R-Y and one frame of color-difference data B-Y arestored in respective field memories 41 and 51 as field image data. Thecolor-difference data that have been stored in the field memories 41 and51 are read out and applied to field memories 42 and 52, respectively.The R-Y color-difference data that have been stored in the framememories 41 and 42 are read out and applied to an adder circuit 45. Ofthe R-Y color-difference data output by the field memory 41, the addercircuit 45 inserts the R-Y color-difference data of the correspondingpixel in the data of the pixel of the missing row. As a result, theadder circuit 45 outputs one frame of R-Y color-difference data. Withregard also to the B-Y difference data, and in a manner similar to thatof the R-Y color-difference data, the data are stored in field memories51 and 52, the data of the corresponding pixel are inserted in the dataof the pixel of the missing row in an adder circuit 55 and one frame ofB-Y color-difference data are outputted by the adder circuit 55.

Thus, one frame of luminance data and R-Y, B-Y color-difference data areobtained.

The above-described processing for generating interpolated image data isapplicable not only to luminance data but also to all types of data suchas both luminance data and color-difference data and all or part of R,G, B color image data (e.g., only G-color image data).

The above-described apparatus for generating frame image data iswell-suited for use in a digital video tape recorder or digital imagedata reproducing apparatus of the kind described below in detail.

FIG. 8 is a block diagram showing the electrical construction of adigital video tape recorder (DVTR) capable of recording and playing backdigital image data. The overall operation of the digital video taperecorder is supervised by a system controller 70.

The digital video tape recorder performs photography in such a mannerthat odd-numbered field images each comprising images on odd-numberedlines and even-numbered field images each comprising images oneven-numbered lines are photographed in alternating fashion, withphotography being repeated at a period of 1/60 of a second.

Shutter speed for photography is decided by so-called electronic shuttercontrol so as to take on an appropriate value (e.g., 1/60 of a second ora shorter time if necessary). A video signal representing the image of asubject is outputted by a CCD 71 every 1/60 of a second, and the videosignal is applied to a CDS (correlated double sampling) circuit 72. TheCDS circuit 72 removes kTC noise components from the video signal, afterwhich the video signal is converted to digital image data by ananalog/digital converter circuit 73. The digital image data are appliedto a gamma-corrector circuit 74, where the data are subjected to a gammacorrection. The gamma-corrected digital image data are applied to afield memory 85 via a data compression circuit 75 (though the data arenot compressed at this time). The data are stored in the field memory 85temporarily.

The image data output by the field memory 85 are applied to the datacompression circuit 75 which, by executing DCT (discrete cosinetransform) processing and quantization processing, subjects the imagedata to data compression. The image data that have been compressed bythe data compression circuit 75 are applied to a field memory 86 throughan error correction code add-on circuit 76 (which merely allows the datato pass). The data are stored in the field memory 86 temporarily.

The image data that have been stored in the field memory 86 aresuccessively applied to the error correction code add-on circuit 76,which proceeds to add on error correction codes.

The image data output by the error correction code add-on circuit 76 areapplied to a recording coding circuit 77. The latter performs coding(e.g., NRZI coding) and delivers its output to a recording/playbackamplifier circuit 87. The image data that have been amplified in therecording/playback amplifier circuit 87 is applied to a magnetic head78. As a result, image data are recorded by the magnetic head 78 in avideo recording area of each track on a magnetic tape 88. Recording ofaudio data and track information also is performed as a matter ofcourse.

The digital video tape recorder shown in FIG. 8 is also capable ofreproducing digital image data that have been stored on the magnetictape 88. Playback modes preferably include a movie playback mode andstill playback mode.

In the mode for playing back digital image data, the image data andother data that have been recorded on the magnetic tape 88 are read outby a magnetic head 91 and applied to the recording/playback amplifiercircuit 87. The data amplified by the recording/playback amplifiercircuit 87 are applied to a demodulator circuit 92. Data demodulation isperformed by the demodulator circuit 92 and the demodulated data areapplied to and temporarily stored in the field memory 86 via an errorcorrection circuit 93. The data that have been recorded in the fieldmemory 86 are read out and applied to an error correction circuit 93. Ifthe data demodulated by the demodulator circuit 92 contain a data error,then error correction processing is executed in the error correctioncircuit 93. The digital image data representing the image of the subjectin the data that have been subjected to error correction processing areapplied to the field memory 85 via a data decompression circuit 94.

In the movie playback mode, the image data that have been stored in thefield memory 85 are applied to the data decompression circuit 94,whereby the compressed image data are subjected to data decompressionprocessing. The data of the odd field images and the data of the evenfield images are alternately applied to a monitor display unit 99 toreproduce a movie. The monitor display unit 99 may be provided on thedigital video tape recorder.

The digital video tape recorder illustrated in FIG. 8 is also capable ofstill-picture playback in addition to movie playback. In thestill-picture playback mode, the image data that have been decompressedin the data decompression circuit 94 are applied to an apparatus 96 forgenerating frame image data.

In the manner described above, the apparatus 96 generates frame imagedata from field image data representing a field image. The generatedframe image data are applied to a printer 98 to print a blur-free,high-quality still picture.

As many apparently widely different embodiments of the present inventioncan be made without departing from the spirit and scope thereof, it isto be understood that the invention is not limited to the specificembodiments thereof except as defined in the appended claims.

What is claimed is:
 1. An apparatus for generating interpolated imagedata in which, when data of consecutive first, second and third fieldimages are given from among a series of image data obtained byinterlaced scanning wherein odd field images comprising pixels on oddlines and even field images comprising pixels on even lines repeat inalternating fashion, interpolated image data are generated representingpixels on even lines in a case where the second field is an odd fieldand pixels on odd lines in a case where the second field is an evenfield in order to convert said second field image, which is locatedbetween the first field image and the third field image, to a frameimage, said apparatus comprising:motion detecting means for detectingwhether motion has occurred between images in windows set atcorresponding positions on respective ones of the first field image andthird field image, wherein said windows are longer vertically thanhorizontally; first interpolated image data generating means which, whenmotion has been detected by said motion detecting means, is forgenerating image data of a pixel at a position corresponding to pixelsin said windows based upon image data of pixels, of the second field,neighboring this pixel from above and below; and second interpolatedimage data generating means which, when no motion has been detected bysaid motion detecting means, is for generating image data of a pixel ata position corresponding to pixels in said windows based upon image dataof a corresponding pixel in at least one of said first field image andsaid third field image.
 2. The apparatus according to claim 1, furthercomprising means for performing control so as to repeat, at eachposition of said windows, motion detection processing by said motiondetecting means and interpolated image data generating processing bysaid first interpolated image data generating means or interpolatedimage data generating processing by said second interpolated image datagenerating means while the positions of said windows are successivelyshifted on the first field image and third field image horizontally andvertically in increments of window size.
 3. The apparatus according toclaim 1, wherein said motion detecting means detects that motion hasoccurred when the value of a sum is greater than a predeterminedthreshold value, wherein said sum is obtained by summing, within saidwindows, differences between absolute values of levels of correspondingpixels in the window of the first field image and the window of thethird field image.
 4. The apparatus according to claim 3, wherein saidthreshold value is a value obtained by multiplying the number of pixelsconstituting said window by a numerical value of 1˜3 when the pixellevel is represented by eight bits.
 5. The apparatus according to claim3, wherein said threshold value is a value obtained by multiplying thenumber of pixels constituting said window by a numerical value of1.5˜2.5 when the pixel level is represented by eight bits.
 6. Theapparatus according to claim 1, wherein the number of pixels verticallyof said windows is twice the number of pixels horizontally.
 7. Theapparatus according to claim 1, wherein said first interpolated imagedata generating means includes:means for extracting image datarepresenting pixels above and below a pixel missing in the second fieldimage, as well as image data representing pixels above and below pixelswhich should be present at positions neighboring the missing pixel inthe horizontal direction, obtaining values of sums of the image datarepresenting these pixels, and determining whether the values of saidsums obtained are increasing or decreasing with regard to pixels in thehorizontal direction; and means for generating data representing themissing pixel using the data representing the pixels above and below themissing pixel when the values of said sums are increasing or decreasing,calculating correlation values between pixels in three directions,namely of pixels above and below and diagonally above and below themissing pixel when the values of said sums are not increasing ordecreasing, and generating data representing the missing pixel usingdata representing pixels which give the highest correlation value.
 8. Amethod of generating interpolated image data in which, when data ofconsecutive first, second and third field images are given from among aseries of image data obtained by interlaced scanning wherein odd fieldimages comprising pixels on odd lines and even field images comprisingpixels on even lines repeat in alternating fashion, interpolated imagedata are generated representing pixels on even lines in a case where thesecond field is an odd field and pixels on odd lines in a case where thesecond field is an even field in order to convert said second fieldimage, which his located between the first field image and the thirdfield image, to a frame image, said method comprising:detecting whetherthere is motion between images in windows set at corresponding positionson respective ones of the first field image and third field imagewherein the windows are longer vertically than horizontally; performingfirst interpolated image data generating processing, when motion hasbeen detected, for generating image data of a pixel at a positioncorresponding to pixels in said windows based upon image data of pixels,of the second field, neighboring this pixel from above and below; andperforming second interpolated image data generating processing, when nomotion has been detected, for generating image data of a pixel at aposition corresponding to pixels in said windows based upon image dataof a corresponding pixel in at least one of said first field image andsaid third field image.
 9. The method according to claim 8, furthercomprising repeating, at each position of said windows, the motiondetection processing and said first interpolated image data generatingprocessing or said second interpolated image data generating processingwhile the positions of said windows are successively shifted on thefirst field image and third field image horizontally and vertically inincrements of window size.
 10. The method according to claim 8, whereinsaid first interpolated image data generating processingincludes:extracting image data representing pixels above and below apixel missing in the second field image, as well as image datarepresenting pixels above and below pixels which should be present atpositions neighboring the missing pixel in the horizontal direction,obtaining values of sums of the image data representing these pixels,and determining whether values of said sums obtained are increasing ordecreasing with regard to pixels in the horizontal direction; andgenerating data representing the missing pixel using the datarepresenting the pixels above and below the missing pixel when thevalues of said sums are increasing or decreasing, calculatingcorrelation values between pixels in three directions, namely of pixelsabove and below and diagonally above and below the missing pixel whenthe values of said sums are not increasing or decreasing, and generatingdata representing the missing pixel using data representing pixels whichgive the highest correlation value.
 11. The method according to claim 8,wherein said detecting includes summing, within said windows,differences between absolute values of levels of corresponding pixels inthe window of the first field image and the window of the third fieldimage and determining that there is motion when a result of said summingis greater than a predetermined threshold value.
 12. The methodaccording to claim 11, wherein said determining includes obtaining thethreshold value by multiplying the number of pixels constituting saidwindow by a numerical value of 1˜3 when the pixel level is representedby eight bits.
 13. The method according to claim 11, wherein saiddetermining includes obtaining the threshold value by multiplying thenumber of pixels constituting said window by a numerical value of1.5˜2.5 when the pixel level is represented by eight bits.
 14. Themethod according to claim 8, wherein the number of pixels vertically istwice the number of pixels horizontally in said windows.
 15. Anapparatus for generating interpolated image data in which, when data ofconsecutive first, second and third field images are given from among aseries of image data obtained by interlaced scanning wherein odd fieldimages comprising pixels on odd lines and even field images comprisingpixels on even lines repeat in alternating fashion, interpolated imagedata are generated representing pixels on even lines in a case where thesecond field is an odd field and pixels on odd lines in a case where thesecond field is an even field in order to convert said second fieldimage, which is located between the first field image and the thirdfield image, to a frame image, said apparatus comprising:motiondetecting means for detecting whether motion has occurred between imagesin windows set at corresponding positions on respective ones of thefirst field image and third field, said motion detecting means detectsthat motion has occurred when the value of a sum is greater than apredetermined threshold value, wherein said sum is obtained by summing,within said windows, differences between absolute values of levels ofcorresponding pixels in the window of the first field image and thewindow of the third field image, wherein said threshold value is a valueobtained by multiplying the number of pixels constituting said window bya numerical value of 1˜3 when the pixel level is represented by eightbits; first interpolated image data generating means which, when motionhas been detected by said motion detecting means, is for generatingimage data of a pixel at a position corresponding to pixels in saidwindows based upon image data of pixels, of the second field,neighboring this pixel from above and below; and second interpolatedimage data generating means which, when no motion has been detected bysaid motion detecting means, is for generating image data of a pixel ata position corresponding to pixels in said windows based upon image dataof a corresponding pixel in at least one of said first field image andsaid third field image.
 16. The apparatus according to claim 16, whereinthe numerical value is between 1.5 and 2.5.
 17. The apparatus accordingto claim 15, further comprising means for performing control so as torepeat, at each position of said windows, motion detection processing bysaid motion detection processing means and interpolated image datagenerating processing by said first interpolated image data generatingprocessing means or interpolated image data generating processing saidsecond interpolated image data generating processing while the positionsof said windows are successively shifted on the first field image andthird field image horizontally and vertically in increments of windowsize.
 18. The apparatus according to claim 1, wherein the number ofpixels vertically of said windows is twice the number of pixelshorizontally.
 19. The apparatus according to claim 15, wherein saidfirst interpolated image data generating means includes:means forextracting image data representing pixels above and below a pixelmissing in the second field image, as well as image data representingpixels above and below pixels which should be present at positionsneighboring the missing pixel in the horizontal direction, obtainingvalues of sums of the image data representing these pixels, anddetermining whether the values of said sums obtained are increasing ordecreasing with regard to pixels in the horizontal direction; and meansfor generating data representing the missing pixel using the datarepresenting the pixels above and below the missing pixel when thevalues of said sums are increasing or decreasing, calculatingcorrelation values between pixels in three directions, namely of pixelsabove and below and diagonally above and below the missing pixel whenthe values of said sums are not increasing or decreasing, and generatingdata representing the missing pixel using data representing pixels whichgive the highest correlation value.